Image processing device capable of generating wide-range image

ABSTRACT

A digital camera according to the present invention includes a CCD, an image display section, and a CPU. The CPU controls the CCD to perform consecutive image-capturing, and generates a wide-range image based on a plurality of images consecutively captured by the CCD. In addition, the CPU detects a predetermined trigger indicating the completion of consecutive image-capturing in a predetermined direction by the CCD. Furthermore, the CPU displays on the image display section information indicating a range to be consecutively captured by the CCD to generate a panoramic image, while changing the information every time the predetermined trigger is detected.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority fromthe prior Japanese Patent Application No. 2011-084383, filed Apr. 6,2011, the entire contents of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing device such as adigital camera or a mobile phone having an imaging function, an imageprocessing method, and a recording medium.

2. Description of the Related Art

In digital cameras, mobile phones having an imaging function, etc., thelimit of a viewing angle is dependent on the hardware specification ofthe device body, such as the focal distance of the lens, the size of theimage sensor, and the like. As a solution to the problem of this viewingangle limit, a conversion lens for wide-range imaging or the like isattached in front of a preexisting lens in an imaging apparatus (forexample, refer to Japanese Patent Application Laid-Open (Kokai)Publication Nos. 2004-191897, 2005-027142, and 2005-057548), or aplurality of lenses are provided in advance and the lenses are switcheddepending on the intended imaging operation (for example, refer toJapanese Patent Application Laid-Open (Kokai) Publication No.2007-081473).

However, in the above-described conventional technologies, theconversion lens for wide-range imaging is required to be attached or thelenses are required to be switched depending on the intended imagingoperation, every time wide-range imaging is performed. Accordingly,there are problems regarding operability and costs. In addition, evenwith conversion lenses for wide-range imaging or switchable lenses, theuser still has difficulty in acquiring a desired wide-range image.

SUMMARY OF THE INVENTION

An object of the present invention is to provide an image processingdevice, an image processing method, and a recording medium capable ofeasily and efficiently acquiring images required to generate awide-range image, without performing lens replacement.

In order to achieve the above-described object, in accordance with oneaspect of the present invention, there is provided an image processingdevice comprising: an imaging section; a display section; an imagingcontrol section which controls the imaging section to performconsecutive image-capturing; a wide-range image generating section whichgenerates a wide-range image based on a plurality of imagesconsecutively captured by the imaging section; a detecting section whichdetects a predetermined trigger indicating completion of consecutiveimage-capturing in a predetermined direction by the imaging section; anda display control section which displays on the display sectioninformation indicating a range to be consecutively captured by theimaging section, while changing the information every time thepredetermined trigger is detected.

The present invention has an advantage in that images required togenerate a wide-range image are easily and efficiently acquired withoutlens replacement.

The above and further objects and novel features of the presentinvention will more fully appear from the following detailed descriptionwhen the same is read in conjunction with the accompanying drawings. Itis to be expressly understood, however, that the drawings are for thepurpose of illustration only and are not intended as a definition of thelimits of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the structure of a digital camera 1according to a first embodiment of the present invention;

FIG. 2 is a conceptual diagram for explaining a normal imaging mode;

FIG. 3 is a conceptual diagram for explaining the panoramic imaging modeof the digital camera 1 according to the first embodiment;

FIG. 4 is a conceptual diagram for explaining movements of the digitalcamera 1 (movements made by the user) in the panoramic imaging mode ofthe digital camera 1 according to the first embodiment;

FIG. 5 is a flowchart for explaining operations of the digital camera 1according to the first embodiment;

FIG. 6 is a flowchart for explaining the operation of consecutiveimage-capture processing by the digital camera 1 according to the firstembodiment;

FIG. 7 is a flowchart for explaining the operation of image compositionprocessing by the digital camera 1 according to the first embodiment;

FIG. 8A to FIG. 8C are conceptual diagrams for explaining the operationof the image composition processing by the digital camera 1 according tothe first embodiment;

FIG. 9 is a flowchart for explaining the operation of consecutiveimage-capture processing by a digital camera 1 according to a secondembodiment;

FIG. 10 is a conceptual diagram for explaining the operation of theconsecutive image-capture processing by the digital camera 1 accordingto the second embodiment;

FIG. 11 is a flowchart for explaining the operation of consecutiveimage-capture processing by a digital camera 1 according to a thirdembodiment;

FIG. 12 is a conceptual diagram for explaining the operation of theconsecutive image-capture processing by the digital camera 1 accordingto the third embodiment;

FIG. 13A and FIG. 13B are conceptual diagrams showing other examples ofmovements of the digital camera 1 during panoramic imaging; and

FIG. 14 is a conceptual diagram showing another display example of animaging frame indicating an imaging range and a movement direction.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The preferred embodiments of the present invention will hereinafter bedescribed with reference to the drawings.

A. First Embodiment

A-1. Configuration of the First Embodiment

FIG. 1 is a block diagram showing the structure of a digital camera 1according to a first embodiment of the present invention. In FIG. 1, thedigital camera 1 includes an imaging lens 2, a lens driving section 3, ashutter-aperture 4, a charge-coupled device (CCD) 5 (imaging section), atiming generator (TG) 6, a unit circuit 7, an image processing section(wide-range image generating section, panoramic image generatingsection, auxiliary composite image generating section), a centralprocessing unit (CPU) 11 (imaging control section, display controlsection, detecting section), a dynamic random access memory (DRAM) 12, amemory 13, a flash memory 14, an image display section 15 (displaysection), a key inputting section 16, a card interface (I/F) 17, and amemory card 18.

The imaging lens 2 includes a focus lens, a zoom lens, and the like, andis connected with the lens driving section 3. The lens driving section 3includes a focus motor and a zoom motor that respectively drive thefocus lens and the zoom lens constituting the imaging lens 2 in anoptical axis direction, and a focus motor driver and a zoom motor driverthat drive the focus motor and the zoom motor in accordance with controlsignals from the CPU 11.

The shutter-aperture 4 includes a driver circuit not shown, and thedriver circuit operates the shutter-aperture 4 in accordance withcontrol signals sent from the CPU 11. This shutter-aperture 4 controlsthe amount of light irradiated from the imaging lens 2. The CCD (imagesensor) 5, which is driven in accordance with timing signals of apredetermined frequency generated by the TG 6, converts the light of asubject projected through the imaging lens 2 and the shutter-aperture 4to electrical signals, and outputs them to the unit circuit 7 as imagingsignals.

The unit circuit 7, which is also driven in accordance with timingsignals of a predetermined frequency generated by the TG 6, includes acorrelated double sampling (CDS) circuit that performs correlated doublesampling on imaging signals outputted from the CCD 5 and holds thesampled imaging signals, an automatic gain control (AGC) circuit thatperforms automatic gain control on the sampled imaging signals, and ananalog-to-digital (A/D) converter that converts theautomatic-gain-controlled analog imaging signals to digital signals.Imaging signals outputted from the CCD 5 are sent to the imageprocessing section 8 through this unit circuit 7, as digital signals.

The image processing section 8, which is also driven in accordance withtiming signals of a predetermined frequency generated by the TG 6,performs image processing of image data sent from the unit circuit 7(pixel interpolation processing, γ-correction, luminosity colordifference signal generation, white balance processing, exposurecorrection processing, etc.), compression and extension processing ofimage data (such as compression and extension in Joint PhotographicExperts Group (JPEG) format, Motion-JPEG [M-JPEG] format or MovingPicture Experts Group (MPEG) format), processing for combining aplurality of captured images, etc.

The CPU 11 is a single-chip microcomputer that controls each section ofthe digital camera 1. In particular, according to the first embodiment,the CPU 11 controls each section such that a plurality of images areconsecutively captured at a predetermined cycle (time interval) whilethe user is moving the digital camera 1, the captured images arecombined in a manner that they are partially overlapped with each other(such as by α-blending) , and a single composite image that appears tohave been captured at a wide angle is generated. The details of thisimage composition will be described hereafter.

The DRAM 12 is used as a buffer memory that temporarily stores imagedata sent to the CPU 11 after being captured by the CCD 5, and as aworking memory of the CPU 11. The memory 13 stores a program necessaryfor the CPU 11 to control each section of the digital camera 1 and datanecessary to control each section, and the CPU 11 performs processingbased on this program. The flash memory 14 and the memory card 18 arerecording media that store image data captured by the CCD 5 and thelike.

The image display section 15 includes a color liquid crystal display(LCD) and a driver circuit thereof. In an imaging stand-by state, theimage display section 15 displays a subject captured by the CCD 5 as areal-time image. Also, the image display section 15 displays a recordedimage that has been read out from the flash memory 14 or the memory card23 and expanded when it is replayed. The key inputting section 16includes a plurality of operation keys, such as a shutter switch (SW), azoom SW, a mode key, a SET key, and a cross-shaped key, and outputsoperation signals based on key operations by the user to the CPU 11. Thememory card 18 is detachably mounted on the card I/F 17 by a card slot(not shown) in the main body of the digital camera 1.

FIG. 2 is a conceptual diagram for explaining a normal imaging mode. Asshown in FIG. 2, when capturing in a normal imaging mode, the digitalcamera 1 can only capture an image with the viewing angle S of theimaging system in the digital camera 1.

FIG. 3 is a conceptual diagram for explaining the panoramic imaging modeof the digital camera 1 according to the first embodiment. FIG. 4 is aconceptual diagram showing movements of the digital camera 1 (movementmade by the user) in the panoramic imaging mode of the digital camera 1according to the first embodiment.

In this mode, the user aims the digital camera 1 at a recording targetlandscape by holding it in the vertical direction such that the longerside of the viewing angle is in the vertical direction, and depresses(half-depression to full depression) the shutter SW at the upper leftend of the recording target landscape. Then, as indicated by the arrowin FIG. 3, the user moves the digital camera 1 towards the right fromthe left end when the shutter SW has been depressed (see state #1 inFIG. 4), and after moving it downward at a predetermined position (seestate #2 in FIG. 4), moves it towards the left at another predeterminedposition (see state #3 in FIG. 4). While the user is making thismovement, the digital camera 1 consecutively captures images atpredetermined timings.

Next, the digital camera 1 generates a first panoramic image from aplurality of images captured in state #1 while the digital camera 1 isbeing moved towards the right from the left end, and after generating asecond panoramic image from a plurality of images captured in state #3while the digital camera 1 is being moved towards the left from theright end, ultimately generates a desired wide-range image (lower sidein FIG. 3) by combining the first panoramic image and the secondpanoramic image. Note that images captured in state #2 while the digitalcamera 1 is being moved downward are not stored, because these imagesare not necessary to generate a panoramic image.

A-2 Operations of the First Embodiment

Next, operations of the above-described first embodiment will bedescribed.

FIG. 5 is a flowchart for explaining operations of the digital camera 1according to the first embodiment. First, when the user half-depressesthe shutter SW (Step S10), the CPU 11 performs auto-focusing (AF) (StepS12). Then, when the user fully depresses the shutter SW (Step S14), theCPU 11 consecutively captures a plurality of images at a predeterminedcycle (time interval) (Step S16). Details of the consecutiveimage-capture processing will be described hereafter.

Here, the user aims the digital camera 1 at the recording targetlandscape by holding it in the vertical direction such that the longerside of the viewing angle is in the vertical direction, and depresses(half-depression to full depression) the shutter SW at the upper leftend of the recording target landscape, as shown in FIG. 3. Then, asindicated by the arrow in FIG. 3, the user moves the digital camera 1towards the right from the left end where the shutter SW has beendepressed (see state #1 in FIG. 4), and after moving it downward at apredetermined position (see state #2 in FIG. 4), moves it towards theleft at another predetermined position (see state #3 in FIG. 4). Next,the CPU 11 judges whether or not the consecutive image-capturing hasbeen completed, or in other words, the panoramic imaging has beencompleted (Step S18). When judged that the consecutive image-capturinghas not been completed, the CPU 11 returns to Step S16 and continues theconsecutive image-capture processing.

Conversely, when judged that the consecutive image-capturing has beencompleted, or in other words, the panoramic imaging has been completed(YES at Step S18), the CPU 11 generates a first panoramic image from aplurality of images captured in state #1 while the digital camera 1 isbeing moved towards the right from the left end, and a second panoramicimage from a plurality of images captured in state #3 while the digitalcamera 1 is being moved towards the left from the right end, andultimately generates a desired wide-range image by combining the firstpanoramic image and the second panoramic image (Step S20). Details ofthe image composition processing will be described hereafter.

FIG. 6 is a flowchart for explaining the operation of the consecutiveimage-capture processing by the digital camera 1 according to the firstembodiment. First, the CPU 11 performs positioning between a currentcaptured image and the preceding captured image (Step S30), and judgeswhether the current state is state #1, state #2, or state #3 (Step S32).

Then, when judged that the current state is state #1 in which the useris moving the digital camera 1 towards the right from the left end(state #1 at Step S32), the CPU 11 stores the current captured image asan image for generating a panoramic image (Step S34). Next, the CPU 11judges whether or not the digital camera 1 has reached a predeterminedposition (in this instance, the right end that is the end position instate #1) (Step S36). When judged that the digital camera 1 has notreached the predetermined position (NO at Step S36), the CPU 11 ends theprocessing without changing the current state #1 and returns to the mainroutine shown in FIG. 5.

Hereafter, until the digital camera 1 reaches the predetermined position(in this instance, the right end that is the end position in state #1),the CPU 11 repeats Step S34 and stores captured images as images forgenerating a panoramic image. Then, when the digital camera 1 reachesthe predetermined position (YES at Step S36), the state transitions fromstate #1 to state #2, and accordingly the CPU 11 changes the currentstate to state #2 (Step S38).

When the current state transitions to state #2 where the user moves thedigital camera 1 downward (state #2 at Step S32), the CPU 11 proceeds toStep S42 without storing a captured image as an image for generating apanoramic image (Step S40), and judges whether or not the digital camera1 has reached a predetermined position (in this instance, the lowerright end that is the end position in state #2) (Step S42). When judgedthat the digital camera 1 has not reached the predetermined position (NOat Step S42), the CPU 11 ends the processing without changing thecurrent state #2 and returns to the main routine shown in FIG. 5.

Hereafter, until the digital camera 1 reaches the predetermined position(in this instance, the lower right end that is the end position in state#2), the CPU 11 continues the capturing without storing captured images.Then, when the digital camera 1 reaches the predetermined position (YESat Step 42), the state transitions from state #2 to state #3, andaccordingly the CPU 11 changes the current state to state #3 (Step S44).

When the current state transitions to state #3 where the user moves thedigital camera 1 towards the left from the lower right (state #3 at StepS32), the CPU 11 stores a captured image as an image for generating apanoramic image (Step S46). Next, the CPU 11 judges whether or not thedigital camera 1 has reached a predetermined position (in this instance,the left end that is the end position in state #3) (Step S48). Whenjudged that the digital camera 1 has not reached the predeterminedposition (NO at Step S48), the CPU 11 ends the processing withoutchanging the current state #3 and returns to the main routine shown inFIG. 5.

Hereafter, until the digital camera 1 reaches the predetermined position(in this instance, the left end that is the end position in state #3),the CPU 11 repeats Step S46 and stores captured images as images forgenerating a panoramic image. Then, when the digital camera 1 reachesthe predetermined position (YES at Step S48), the CPU 11 ends theconsecutive image-capturing (Step S50).

Consequently, the plurality of images captured in state #1 while theuser is moving the digital camera 1 towards the right from the left end,and the plurality of images captured in state #3 while the user ismoving the digital camera 1 towards the left from the lower right areacquired. Next, a method for ultimately acquiring a wide-range imageusing these captured images will be described.

FIG. 7 is a flowchart for explaining the operation of the imagecomposition processing by the digital camera 1 according to the firstembodiment. FIG. 8A to FIG. 8C are conceptual diagrams for explainingthe operation of the image composition processing by the digital camera1 according to the first embodiment. First, the CPU 11 acquires an imagefor generating a panoramic image (Step S60), and identifies the state atthe time of the capturing of this image (Step S62). When the state atthe time of the capturing of the image is state #1, the CPU 11 performsimage composition processing for generating a panoramic image #1 on thisimage (Step S64).

Next, the CPU 11 judges whether or not the panoramic image compositionhas been completed (Step S68). When judged that the panoramic imagecomposition has not been completed, the CPU 11 returns to Step S60.Hereafter, the panoramic image #1 is generated by a plurality of imagesFR1 to FR6 captured in state #1 being combined in a manner that they arepartially overlapped with each other (such as by α-blending) as shown inFIG. 8A.

On the other hand, when the acquired image for generating a panoramicimage is an image captured in state #3, the CPU 11 performs imagecomposition processing for generating a panoramic image #2 on this image(Step S66). Then, the CPU 11 judges whether or not the panoramic imagecomposition has been completed (Step S68). When judged that thepanoramic image composition has not been completed, the CPU 11 returnsto Step S60. Hereafter, the panoramic image #2 is generated by aplurality of images FL1 to FL6 captured in state #3 being combined in amanner that they are partially overlapped with each other (such as byα-blending) as shown in FIG. 8B.

When judged that the panoramic image composition has been completed (YESat Step S68), the CPU 11 combines a predetermined area on the lower sideof the panoramic image #1 and a predetermined area on the upper side ofthe panoramic image #2 in a manner that they are partially overlappedwith each other (such as by α-blending), and thereby generates a singlewide-range image (Step S70), as shown in FIG. 8C.

Note that a configuration may be adopted in the above-described firstembodiment in which, when the imaging range of a wide-range image isspecified in advance using the key inputting section 16, a panoramicimage of a size based on the specified imaging range is generated. Also,a configuration may be adopted in which the imaging ranges of panoramicimages to be combined are compared in the image processing section 8, alarger imaging range is adjusted to coincide with the other imagingrange, and then the panoramic images are combined to ultimately generatea wide-range image.

According to the above-described first embodiment, images required togenerate a wide-range image are easily and efficiently acquired withoutlens replacement.

B. Second Embodiment

Next, a second embodiment of the present invention will be described.

The second embodiment is characterized in that the change of movementdirection from state #1 to state #2 or state #2 to state #3 when thedigital camera 1 is moved by the user during panoramic imaging isdetected using a camera-based factor, such as a movement amount andinformation from a direction sensor or an acceleration sensor, or auser-based factor, such as a movement-direction designating operation, ashutter key operation, a user gesture, or sound, as a trigger for thechange.

The structure of a digital camera 1 according to the second embodimentis the same as that in FIG. 1, and therefore explanations thereof areomitted. In addition, the main routine in the panoramic imaging mode isthe same as that in FIG. 5, and the image composition processing is thesame as that in FIG. 7. Therefore, explanations thereof are alsoomitted.

FIG. 9 is a flowchart for explaining the operation of consecutiveimage-capture processing by the digital camera 1 according to the secondembodiment. FIG. 10 is a conceptual diagram for explaining the operationof the consecutive image-capture processing by the digital camera 1according to the second embodiment.

In the second embodiment, a flag that is inverted every time a triggeris detected and a state #N (coefficient) that is incremented by oneevery time a trigger is detected are provided to determine whether thedigital camera 1 is currently in state #1, state #2, or state #3 inresponse to trigger detection. The initial value of the flag is “0”, andthe flag is inverted every time a trigger is detected. The initial valueof the state #N is “1”, and the state #N is a coefficient indicatingwhether the digital camera 1 is currently in state #1, state #2, orstate #3. Until a first trigger is detected, the state #N is “1” andindicates state #1. When it is detected, the state #N becomes “2” andindicates state #2. Then, when the next trigger is detected, the state#N becomes “3” and indicates state #3.

First, the CPU 11 performs positioning between a current captured imageand the preceding captured image (Step S80), and judges whether or not atrigger has been detected (Step S82). When judged that a trigger has notbeen detected, the CPU 11 judges that the digital camera 1 has nottransitioned from state #1 to state #2, or in other words, the digitalcamera 1 is in state #1 where it is being moved towards the right fromthe left end, as shown in FIG. 10. Then, the CPU 11 judges whether ornot the flag is “1” (Step S88). In this case, since a trigger has notbeen detected, the flag is “0” and the state #N is “1”.

Then, since the flag is “0” (NO at Step S88), the CPU 11 stores thecurrent captured image as an image for generating a panoramic image inthe state #N (=1) (Step S90), and after ending the processing, returnsto the main routine shown in FIG. 5. Hereafter, until a trigger isdetected, the CPU 11 repeats Step S90 and stores captured images asimages for generating a panoramic image in the state #N (=1).

Next, in the transition from state #1 to state #2, when the digitalcamera 1 is detected to have reached the end of state #1 (when thecompletion of consecutive image-capturing in a predetermined directionis detected) by a camera-based factor such as a movement amount andinformation from a direction sensor, or a user-based factor such as amovement direction, a shutter key operation, a user gesture, or sound,the CPU 11 detects this factor as a trigger, as shown in FIG. 10 (YES atStep S82). When the trigger is detected, the CPU 11 inverts the flag(Step S84) and sets the state #N to N+1 (Step S86). In this case, theflag becomes “1” and the state #N becomes “2”.

Next, the CPU 11 judges whether or not the flag is “1” (Step S86). Inthis case, since the flag is “1”, the CPU 11 ends the processing withoutstoring a captured image as an image for generating a panoramic image(Step S92), and returns to the main routine shown in FIG. 5. Hereafter,until the next trigger is detected (until the movement direction of thedigital camera enters state #3), the CPU 11 repeatedly performs theoperations of ending the processing and returning to the main routineshown in FIG. 5 without storing a captured image. Accordingly, imagescaptured in state #2 are not stored.

Next, in the transition from state #2 to state #3, when the digitalcamera 1 is detected to have reached the end of state #2 (when thecompletion of consecutive image-capturing in a predetermined directionis detected) by a camera-based factor such as a movement amount andinformation from a direction sensor, or a user-based factor such as amovement direction, a shutter key operation, a user gesture, or sound,the CPU 11 detects this factor as a trigger, as shown in FIG. 10 (YES atStep S82). When the trigger is detected, the CPU 11 inverts the flag(Step S84) and sets the state #N to N+1 (Step S86). In this case, theflag becomes “0” and the state #N becomes “3”.

Next, the CPU 11 judges whether or not the flag is “1” (Step S86). Inthis case, since the flag is “0”, the CPU 11 stores a captured image asan image for generating a panoramic image in the state #N (=3) (StepS90), and after ending the processing, returns to the main routine shownin FIG. 5. Hereafter, until the next trigger is detected, the CPU 11repeats Step S90 and stores captured images as images for generating apanoramic image in the state #N (=3).

Consequently, the plurality of images captured in state #1 while theuser is moving the digital camera 1 towards the right from the left end,and the plurality of images captured in state #3 while the user ismoving the digital camera 1 towards the left from the lower right areacquired.

Next, using the same method as the above-described first embodiment (seeFIG. 7), a panoramic image #1 is generated by the plurality of imagescaptured in state #1 being combined, and a panoramic image #2 isgenerated by the plurality of images captured in state #3 beingcombined. Then, a predetermined area on the lower side of the panoramicimage #1 and a predetermined area on the upper side of the panoramicimage #2 are combined in a manner that they are partially overlappedwith each other (such as by α-blending), whereby a single wide-rangeimage is generated.

According to the above-described second embodiment, images required togenerate a wide-range image are easily and efficiently acquired withoutlens replacement.

C. Third Embodiment

Next, a third embodiment of the present invention will be described.

In the above-described first and second embodiments, the user aims thedigital camera 1 at a recording target landscape by holding it in thevertical direction such that the longer side of the viewing angle is inthe vertical direction, and depresses (half-depression to fulldepression) the shutter SW at the upper left end of the recording targetlandscape. Then, in order to capture all images required to acquire awide-range composite image, the user moves the digital camera 1 towardsthe right from the left end at which the shutter SW has been depressed(state #1), and after moving it downward at a predetermined position(state #2), moves it towards the left at another predetermined position(state #3), as indicated by the arrow in FIG. 3. However, the usercannot easily know how to move the digital camera 1 or whether or notthe required images have been unfailingly acquired.

For this reason, in the third embodiment, when the user depresses theshutter SW in the panoramic imaging mode, an imaging frame indicating anarea that should be captured by the digital camera 1 and a movementdirection indicating in which direction the digital camera 1 should bemoved are displayed on the image display section 15, whereby the user isguided. Also, in the panoramic imaging mode, an image currently formedin the CCD 5 of the digital camera 1 is displayed on the image displaysection 15 as a preview image (low resolution). In addition, a compositeimage generated using preview images is semi-transparently (50%transparency) displayed on the image display section 15.

As described above, in the panoramic imaging mode, an imaging frameindicating an imaging area to be captured next, a movement directionindicating which direction the digital camera 1 should be moved, and areduced image generated from combined captured images are displayed onthe image display section 15. Therefore, the user can easily know inwhich direction the digital camera 1 should be moved.

Note that the structure of the digital camera 1 according to the thirdembodiment is the same as that in FIG. 1, and therefore explanationsthereof are omitted. In addition, the main routine in the panoramicimaging mode is the same as that in FIG. 5, and the image compositionprocessing is the same as that in FIG. 7. Therefore, explanationsthereof are also omitted.

FIG. 11 is a flowchart for explaining the operation of consecutiveimage-capture processing by the digital camera 1 according to the thirdembodiment. FIG. 12 is a conceptual diagram for explaining the operationof the consecutive image-capture processing by the digital camera 1according to the third embodiment.

First, the CPU 11 performs positioning between a current captured imageand the preceding captured image (Step S100), and judges whether thecurrent state is state #1, state #2, or state #3 (Step S102). Whenjudged that the current state is state #1 where the user is moving thedigital camera 1 towards the right from the left end (state #1 at StepS102), the CPU 11 displays an imaging frame FR1 and a movement directionM1 as shown in FIG. 12 (Step S104), and stores the current capturedimage as an image for generating a panoramic image in state #1 (StepS106). Then, the CPU 11 generates a simplified composite image IMG fromreduced images of images captured and stored up to this point, andsemi-transparently (50% transparency) displays the simplified compositeimage IMG (Step S108).

Next, the CPU 11 judges whether or not the digital camera 1 has reacheda predetermined position P1 (in this instance, the end position in state#1; see FIG. 12) (Step S110). When judged that the digital camera 1 hasnot reached the predetermined position P1 (NO at Step S110), the CPU 11ends the processing without changing the current state #1, and returnsto the main routine.

Hereafter, until the digital camera 1 reaches the predetermined positionP1, the CPU 11 repeats Step S106 and Step S108, and semi-transparently(50% transparency) displays an updated composite image every time itstores a captured image as an image for generating a panoramic image instate #1. Then, when the digital camera 1 reaches the predeterminedposition P1 (YES at Step S110), the state transitions from state #1 tostate #2, and accordingly the CPU 11 changes the current state to state#2 (Step S112).

Next, when the current state transitions to state #2 where the usermoves the digital camera 1 downward from the end position in state #1(state #2 at Step S102), the CPU 11 displays an imaging frame FR2 and amovement direction M2 as shown in FIG. 2 (Step S114), and proceeds toStep S118 without storing a captured image as an image for generating apanoramic image (Step S116). Then, the CPU 11 judges whether or not thedigital camera 1 has reached a predetermined position P2 (in thisinstance, the end position in state #2; see FIG. 12) (Step S118). Whenjudged that the digital camera 1 has not reached the predeterminedposition P2 (NO at Step S118), the CPU 11 ends the processing withoutchanging the current state #2 and returns to the main routine shown inFIG. 5.

Hereafter, until the digital camera 1 reaches the predetermined positionP2, the CPU 11 continues the capturing without storing captured images.Then, when the digital camera 1 reaches the predetermined position P2(YES at Step 118), the state transitions from state #2 to state #3, andaccordingly the CPU 11 changes the current state to state #3 (StepS120).

Next, when the current state transitions to state #3 where the usermoves the digital camera 1 towards the left from state #2 (state #3 atStep S102), the CPU 11 displays an imaging frame FR3 and a movementdirection M3 as shown in FIG. 12 (Step S122), and stores a capturedimage as an image for generating a panoramic image in state #3 (StepS124). Then, the CPU 11 generates a simplified composite image fromreduced images of images captured and stored up to this point, andsemi-transparently (50% transparency) displays the simplified compositeimage (Step S126).

Hereafter, until the digital camera 1 reaches a predetermined positionP3 (endpoint), the CPU 11 repeats Step S124 and Step S126, andsemi-transparently (50% transparency) displays an updated compositeimage every time it stores a captured image as an image for generating apanoramic image in state #3. Then, when the digital camera 1 reaches thepredetermined position P3 (YES at Step S128), the CPU 11 ends theconsecutive image-capturing (Step S130).

Consequently, the plurality of images captured in state #1 while theuser is moving the digital camera 1 towards the right from the left end,and the plurality of images captured in state #3 while the user ismoving the digital camera 1 towards the left from the lower right areacquired.

Next, using the same method as the above-described first and secondembodiments (see FIG. 7), a panoramic image #1 is generated by theplurality of images captured in state #1 being combined, and a panoramicimage #2 is generated by the plurality of images captured in state #3being combined. Then, a predetermined area on the lower side of thepanoramic image #1 and a predetermined area on the upper side of thepanoramic image #2 are combined in a manner that they are partiallyoverlapped with each other (such as by α-blending), whereby a singlewide-range image is generated.

Note that, in the above-described third embodiment, it is preferablethat a margin MG (extra space) is provided in the imaging frames FR1 toFR3 displayed on the image display section 15, as shown in FIG. 12. Thatis, the imaging frames should be slightly larger than an image area thatis actually captured. As a result of the margin MG being provided, extraspace is given to the range in which the digital camera 1 is moved,whereby stress on the user moving the digital camera 1 is reduced.

In addition, although the change of movement direction from state #1 tostate #2 or state #2 to state #3 is judged based on whether or not apredetermined position has been reached, the judgment criterion is notlimited thereto, and it may be judged using a camera-based factor, suchas a movement amount and information from a direction sensor or anacceleration sensor, or a user-based factor, such as amovement-direction designating operation, a shutter key operation, auser gesture, or sound, as a trigger for the change, as in theabove-described second embodiment.

According to the above-described third embodiment, a composite image isdisplayed on the image display section 15 in real-time. In addition, animaging frame and a movement direction for moving the digital camera 1are also displayed. Therefore, the user is only required to move thedigital camera 1 while viewing the imaging frame and the movementdirection. As a result, a plurality of images required to generate awide-range image that cannot be acquired by a single image-captureoperation is easily and efficiently captured, and the wide-range imageis easily generated.

Note that, although the digital camera 1 is moved from left to right,top to bottom, and right to left in the panoramic imaging mode in theabove-described first to third embodiments, the movement is not limitedthereto. For example, a configuration may be adopted in which aplurality of images is captured while the digital camera 1 is beingmoved in one direction, and a single wide-range image is generated bythese images being combined, as shown in FIG. 13A. Alternatively, aconfiguration may be adopted in which the movement of the digital camera1 in the horizontal direction is repeated three times or more, such asfrom left to right, top to bottom, right to left, top to bottom, andleft to right, as shown in FIG. 13B.

Also note that, although the imaging frame indicating an imaging areaand the movement direction are displayed in the above-described thirdembodiment, it is not limited thereto, and a configuration may beadopted in which captured portions 30 of the overall frame 20 of awide-range image that is ultimately generated are colored.

While the present invention has been described with reference to thepreferred embodiments, it is intended that the invention be not limitedby any of the details of the description therein but includes all theembodiments which fall within the scope of the appended claims.

What is claimed is:
 1. An image processing device comprising: an imagingdevice; a display; and a CPU which is configured to function as: animaging control section which controls the imaging device to performconsecutive image-capturing; a wide-range image generating section whichgenerates a wide-range image based on a plurality of imagesconsecutively captured by the imaging device; a detecting section whichdetects a predetermined trigger indicating completion of consecutiveimage-capturing in one of a plurality of directions by the imagingdevice; a first display control section which displays, on the displaysection, at least one of a first range to be captured by the imagingdevice and a first direction for moving the image processing device; anda second display control section which newly displays, on the displaysection, at least one of a second range to be captured by the imagingdevice and a second direction for moving the image processing devicesuch that the at least one of the first range and the first direction ischanged to the at least one of the second range and the seconddirection, respectively, in response to the predetermined trigger beingdetected by the detecting section, wherein the second range and thesecond direction are different from the first range and the firstdirection, respectively.
 2. The image processing device according toclaim 1, wherein the CPU is further configured to function as apanoramic image generating section that generates a plurality ofpanoramic images by combining the plurality of images captured inrespective ones of the plurality of directions by the imaging device;wherein the wide-range image generating section generates the wide-rangeimage by combining the plurality of panoramic images generated by thepanoramic image generating section.
 3. The image processing deviceaccording to claim 2, wherein the detecting detecting section detects aplurality of predetermined triggers indicating completion of consecutiveimage-capturing in respective ones of the plurality of directions by theimaging device, and the panoramic image generating section generates oneof the plurality of panoramic images each time the detecting sectiondetects one of the plurality of predetermined triggers.
 4. The imageprocessing device according to claim 1, wherein: the first displaycontrol section displays, on the display, a first frame indicating thefirst range, and the second display control section displays, on thedisplay, a second frame indicating the second range.
 5. The imageprocessing device according to claim 1, wherein the CPU is furtherconfigured to function as: a composite image generating section whichgenerates a composite image corresponding to the wide-range image to begenerated by combining the plurality of images consecutively captured bythe imaging device; wherein the first display control section and thesecond display control section respectively display on the display, atleast one of the first range and the first direction, and at least oneof the second range and the second direction, along with the compositeimage generated by the composite image generating section.
 6. The imageprocessing device according to claim 1, wherein the detecting sectiondetects, as the predetermined trigger, either of a movement amount,information from a direction sensor, a change in movement direction, apredetermined instruction operation by a user, an instruction by soundfrom the user, or an instruction by movement by the user.
 7. An imageprocessing method for an image processing device comprising: performingconsecutive image-capturing by an imaging device; generating awide-range image based on a plurality of images consecutively capturedby the imaging device; detecting a predetermined trigger indicatingcompletion of consecutive image-capturing in one of a plurality ofdirections by the imaging device; first displaying on a display at leastone of a first range to be captured by the imaging device and a firstdirection for moving the image processing device; and second displayingto newly display, on the display, at least one of a second range to becaptured by the imaging device and a second direction for moving theimage processing device such that the at least one of the first rangeand the first direction is changed to the at least one of the secondrange and the second direction, respectively, in response to thepredetermined trigger being detected, wherein the second range and thesecond direction are different from the first range and the firstdirection, respectively.
 8. A non-transitory computer-readable storagemedium having stored thereon a program that is executable by a computerof an image processing device that performs image processing, theprogram being executable by the computer to perform functionscomprising: controlling an imaging device to perform consecutiveimage-capturing; generating a wide-range image based on a plurality ofimages consecutively captured by the imaging device; detecting apredetermined trigger indicating completion of consecutiveimage-capturing in one of a plurality of directions by the imagingdevice; first displaying on a display at least one of a first range tobe captured by the imaging device and a first direction for moving theimage processing device; and second displaying to newly display, on thedisplay, at least one of a second range to be captured by the imagingdevice and a second direction for moving the image processing devicesuch that the at least one of the first range and the first direction ischanged to the at least one of the second range and the seconddirection, respectively, in response to the predetermined trigger beingdetected, wherein the second range and the second direction aredifferent from the first range and the first direction, respectively.